The present invention relates to testing computer systems, and more specifically, to system architecture validation.
Thorough pre-silicon testing of integrated circuits and computer systems is an essential step in obtaining a production quality design. However, there are hindrances to the successful validation of increasingly-complex designs: difficulty in discovering subtle errors in simulation results, the lack of consistent and comprehensive checking methodologies for complex transaction scenarios, and the extensive time and experience required for quality simulation analysis.
To effectively validate a system, simulation results must be analyzed. The results should be analyzed in real time or captured for post-simulation analysis. During the simulation, all system activity is recorded in log files called trackers. These trackers, located at each transaction interface, provide a macroscopic view of bus transactions. The host tracker, for example, lists all processor bus activity as well as second-level cache cycles, inquire/snoop cycles, and system interrupts. Generally, trackers are located at all system transaction and event interfaces, including main memory (DRAM), industry standard architecture (ISA) bus, and peripheral component interconnect (PCI) bus. Once simulation events have been recorded for a particular test suite, they must be evaluated for correctness.
One prior art method of evaluation is to obtain an initial xe2x80x9cgoodxe2x80x9d simulation. This is done by actually visually inspecting the output of an initial simulation and determining its accuracy. For each type of test, an initial good simulation must be performed and inspected.
Further simulations are compared against the results of the known good simulation. There are several weaknesses with this approach. First, although one assumes that the prior xe2x80x9cgoodxe2x80x9d simulation is free of bugs, it may still contain errors that were missed by the visual inspection. Second, methods for comparing the results of the initial good simulation and the current simulation must be sophisticated (as opposed to the simple UNIX xe2x80x9cdiffxe2x80x9d program) to provide for allowable differences, such as system latency, but detect subtle differences like single bit errors. A subtle error may pass undetected during an early test and propagate through subsequent testing because its xe2x80x9cdiffxe2x80x9d is clean.
An example of comparing an initial good simulation with simulation runs was performed. Bus trackers were generated and stored for each of the suite of seven-hundred and fifty regression tests. A variant of the UNIX xe2x80x9cdiffxe2x80x9d program was created to compare the results of each test to prior xe2x80x9cknown goodxe2x80x9d results. This tedious, error prone process required one week for an engineer to review all the observed differences. On several occasions, the xe2x80x9cknown goodxe2x80x9d results were found to have errors, and differences that showed bugs were ignored.
Thus, a tool that consistently analyzes simulation results in an automated fashion independent of the results of previous simulation iterations would be advantageous.
A method and apparatus for transaction checking for system architecture validation are described. Tracking data is received from trackers in the system. The tracking data is parsed to construct queues. These queues are compared with each other. For one embodiment, the queues are further compared with predicted behavior of the element that was tested. Discrepancies between the queues and the predicted behavior are flagged.